Stereo cameras, i. e. camera devices including at least two cameras arranged in a certain mutual distance, in order to obtain three-dimensional image information, have a wide field of application. One field of use relates to the monitoring of scenes, where stereo cameras provide additional information, namely about the three-dimensional position and/or extension of captured objects. Usually, the corresponding stereo cameras are attached to a building structure such as a ceiling or wall, in a fixed position and orientation that allows for monitoring the respective scene. Three dimensional capturing of scenes is particularly useful in the field of people counting, where the additional information allows for higher precision, especially when monitoring complex or extended scenes with many objects.
In order to obtain reliable results, the stereo cameras need to be calibrated. After calibration, the stereo camera (including the electronics for the processing of image data obtained by the at least two cameras) will yield at least two rectified images, i. e. images where distortion effects due to the used lenses and further optical elements are compensated, and where the coordinates of the images obtained by the cameras correspond to each other with the exception of a disparity along the first direction caused by the corresponding distance between the location of the cameras.
Usually, the stereo cameras are calibrated in the manufacturing process (factory calibration). This is usually done by employing precisely measured and positioned targets (often checkerboard patterns). However, due to transport, thermal effects, aging etc. calibration errors may be caused that impair the quality of the stereo images or of results obtained from analysis of the stereo images. In particular, lost calibration may lead to an impaired quality of the dense stereo image due to mismatches of the stereo algorithm and to defective reprojection into the three-dimensional space, which leads inter alia to erroneous distance measurements.
In order to reestablish the desired accuracy, the device needs to be calibrated anew. With known devices, this usually means that they will need to be sent back to the factory where the required equipment for performing the recalibration is available. This is expensive and cumbersome. Furthermore, doing so is not viable with respect to devices that are in continuous use, in particular safety relevant devices.
US 2015/022669 A1 (Microsoft) relates to real-time registration of a camera array in an image capture device. To do so, a selected, relatively small subset of independent parameters in a mapping function is adjusted. This allows for performing the re-calibration with a reduced number of matching patterns, i. e. based on images that are captured during normal camera array usage, without the need for specialized calibration targets.
The feasibility of this method depends on whether it is possible to identify the small subset of independent parameters that a) allows for field recalibration in the described manner and b) allows for the correction of all the calibration errors that are to be expected after factory calibration. In connection with certain stereo cameras and the calibration errors to be expected, the conditions a) and b) may not be simultaneously fulfilled, making this method unfeasible.
JP 2013-113600 (Sharp Corp.) discloses another approach. It relates to a method for recorrecting the calibration of a stereo camera. The calibration corrects distortions and inclinations of the images. A calibration error is determined by identifying matching spots of the images (preprocessed using the present calibration parameters). Based on the identified calibration error, offset values for compensating the error are generated.
However, in the context of typical scenes captured by stereo cameras, e. g. in the field of object or people counting, these offsets are not sufficently comprehensive and precise to reliably allow for correcting all possible calibration errors. This is especially true in connection with lenses exhibiting substantial distortion and having a large field-of-view (FOV).